Systems and methods for detecting a press on a touch-sensitive surface

ABSTRACT

Systems and methods for displaying and intuitively interacting with keyboards on a touch-sensitive surface are disclosed herein. In one aspect, a method is performed at an electronic device with one or more processors, memory, a touch-sensitive display, and one or more touch sensors coupled to the touch-sensitive display. The method includes: displaying a plurality of keys on a keyboard on the touch-sensitive display and detecting, by the one or more touch sensors, a first contact at a first key of the plurality of keys on the keyboard. The method further includes: determining a value of a signal corresponding to the first contact. When the value is above a first non-zero threshold, the method includes actuating the first key. When the value is between a second non-zero threshold and the first non-zero threshold, the method includes forgoing actuating the first key.

RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/308,428, filed Nov. 30, 2011, which is acontinuation-in-part of U.S. patent application Ser. No. 13/171,124,filed Jun. 28, 2011, which claims priority to U.S. ProvisionalApplication No. 61/472,799, filed Apr. 7, 2011, U.S. ProvisionalApplication No. 61/418,279, filed Nov. 30, 2010, and U.S. ProvisionalApplication No. 61/359,235, filed Jun. 28, 2010, and is acontinuation-in-part of U.S. patent application Ser. No. 12/234,053,filed Sep. 19, 2008, now U.S. Pat. No. 8,325,141, which claims priorityto U.S. Provisional Application No. 60/973,691, filed Sep. 19, 2007,each of which is incorporated by reference in its entirety. Thisapplication is also a continuation-in-part of U.S. patent applicationSer. No. 13/308,416, filed Nov. 30, 2011, which is acontinuation-in-part of U.S. patent application Ser. No. 13/171,124,filed Jun. 28, 2011, which claims priority to U.S. ProvisionalApplication No. 61/472,799, filed Apr. 7, 2011, U.S. ProvisionalApplication No. 61/418,279, filed Nov. 30, 2010, and U.S. ProvisionalApplication No. 61/359,235, filed Jun. 28, 2010, and is acontinuation-in-part of U.S. patent application Ser. No. 12/234,053,filed Sep. 19, 2008, now U.S. Pat. No. 8,325,141, which claims priorityto U.S. Provisional Application No. 60/973,691, filed Sep. 19, 2007,each of which is incorporated by reference in its entirety. Thisapplication is also a continuation-in-part of U.S. patent applicationSer. No. 13/442,855, filed Apr. 10, 2012, which is acontinuation-in-part of U.S. patent application Ser. No. 12/234,053,filed Sep. 19, 2008, now U.S. Pat. No. 8,325,141, which claims priorityto U.S. Provisional Application No. 60/973,691, filed Sep. 19, 2007,each of which is incorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates to a touch-sensitive surface that allows auser to rest their hands or fingers on the touch-sensitive surfacewithout causing an event actuation and, more specifically, thetouch-sensitive surface is configured to the user to seamlessly rest,tap, and press on an onscreen keyboard displayed on the touch-sensitivesurface.

BACKGROUND

The origin of the modern keyboard as the primary method for inputtingtext and data from a human to a machine dates back to early typewritersin the 19th century. As computers were developed, it was a naturalevolution to adapt the typewriter keyboard to be used as the primarymethod for inputting text and data. While the implementation of the keyson a typewriter and, subsequently, computer keyboards have evolved frommechanical to electrical and finally to electronic, the size, placement,and mechanical nature of the keys themselves have remained largelyunchanged.

As computers evolved and graphical user interfaces developed, the mousepointer became a common user input device. With the introduction ofportable “laptop” computers, various new pointing devices were inventedas an alternative to the mouse, such as trackballs, joysticks, andtouchpads (also referred to “trackpads”). The overwhelming majority oflaptop computers now incorporate the touchpad as the primary pointingdevice.

Prior to computers, a common office instrument used for performingnumerical calculations was the “adding machine.” This deviceincorporated number keys along with common mathematical operation keys,such as add, subtract, multiply, and divide. The operator performed dataentry using these machines, which then displayed the result, printed theresult, or both. Experienced adding machine operators were able tomemorize the location of the keys and enter data and perform operationsvery quickly without looking. As computers became common, the need forefficient numeric entry persisted and the “adding machine” functionswere added to computer keyboards in the form of a numeric keyboard (or“numpad”) typically located to the right of the standard keyboard.

Combining the three primary user interface devices of keyboard,touchpad, and numpad into a single device results in the device becomingunreasonably large. The problem is further complicated by the fact thatmany modern keyboards incorporate yet additional keys for pagenavigation, multimedia controls, gaming, and keyboard settingsfunctions. The result can be a “keyboard” that is often larger than thecomputer itself.

Further, a new form of portable computing device has recently emerged,commonly referred to as a “tablet” computer. This type of portablecomputing device typically does not have an integrated keyboard, relyinginstead solely on touch as the primary means of human-computerinterface. Many believe tablets and, eventually, “touch surfaces” thatare integrated into daily life will become the standard way humans willinterface with “computers” in the future.

While this new paradigm of touch-centric computing has many advantages,one marked disadvantage is the lack of a keyboard. Although externalphysical keyboards can typically be connected to touch-screen computers,it often defeats the purpose of the device and negates its advantagesover traditional laptop computers.

As the evolution of computing devices has progressed toward touch-baseduser interfaces, a natural evolution for the idea of a keyboard has beento carry it into the virtual world of the computer display by designingonscreen keyboards. Smaller touchscreen devices such as PDAs andSmartphones don't have sufficient screen size to allow people to type onan onscreen keyboard using the conventional method of touch-typing withmultiple fingers. As a result, a plethora of inventions have sought toprovide alternative text input methods that require less physical spacethan a conventional keyboard layout. While these inventions have varyingbenefits for entering text on a small onscreen keyboard, they don'tallow text entry at speeds that compare to standard “ten-finger” typingon a conventional keyboard.

Thus, it is desirable to find a yet faster way for entering text thatmore closely matches the typing style learned on conventional keyboards.In doing so, there are three primary challenges: first, overcoming therelatively large amount of display real estate required for a 10-fingeronscreen keyboard. Second, overcoming the lack of tactile feedbackcommon in mechanical keyboards. And third, allowing the user to resttheir fingers on the “home-row” position on the onscreen keyboard, asthey normally would on a conventional electromechanical keyboard.

Traditionally, these touch sensitive surfaces respond immediately to theuser's touch (or release). The paradigm is simple: point, touch, select.While this works well for many applications, it is problematic insituations where the user desires to rest their hands and/or fingers onthe surface. A touch-sensitive keyboard (onscreen or stand-alone) is agood example of such a situation; a trained ten-finger touch typistrelies on resting their fingers on the home row of the keyboard and thenpressing keys to initiate an action. On traditional touch-sensitivesurfaces, this isn't possible because as soon as the user touches thesurface to rest their fingers, an action is initiated. These solutionsdon't take into account the need for the user to rest theirhands/fingers on the surface.

SUMMARY

Disclosed herein are methods and systems that solve the space problem byintegrating the numeric keypad part of the keyboard and the touchpad inthe same physical location (e.g., a touch-sensitive surface). Thissurface can be used to provide all the functions of the keyboard,numpad, and touchpad, but in a much smaller space since it makes itpossible to “multiplex” or use the same physical space on the surfacefor multiple functions. The touch surface may incorporate either adynamic or static display beneath it, or a mixture of both.

In one aspect, the numeric keypad and the touchpad occupy the samephysical space. This is possible due to the fact that thetouch-sensitive surface, unlike traditional mechanical keys, can havethe spacing, size, orientation, and function of its “keys” dynamicallyassigned.

In another aspect, the system has three modes of operation: numpad mode,touchpad mode, and auto-detect mode. A visual indicator communicateswith the user which mode it is in. The user changes the mode viaactivation of a key or combination of keys on the keyboard. Visualindicators provide feedback to the user as to which mode the device isin.

In a further aspect, the system automatically determines which mode theuser intends based on their interaction with the touch surface. Forexample, if the user slides their finger across the surface, they mostlikely intend for it to act as a touchpad, causing the pointer to move.Similarly, if the user taps their finger on a specific sector of thetouch surface assigned to a number key, then they most likely intend forit to be used as a numpad.

In some embodiments, the system includes a surface with a multi-modearea, a plurality of touch sensors coupled to the surface, a pluralityof motion sensors, and a processor in signal communication with thesurface, the plurality of touch sensors, and the plurality of motionsensors is provided. The plurality of touch sensors are configured togenerate at least one sense signal based on sense user contact with thesurface. The plurality of motion sensors are configured to generate amotion signal based on sensed vibrations of the surface. The processoris configured to determine mode of operation associated with themulti-mode area based on interpretation of at least one of the generatedat least one sense signal and the motion signal associated with themulti-mode area.

In some embodiments, the modes of operation include at least two of akeyboard mode, a numeric keypad mode, or a touchpad mode.

In some embodiments, the processor is further configured to determinethe mode of operation based on a signal associated with a userselection.

In some embodiments, the surface includes a display device coupled tothe processor. In some embodiments, the user selection includesactivation of a mode key displayed by the processor on the surface.

In some embodiments, the surface includes at least one visual indicatorand the processor illuminates the at least one visual indicator based onthe determined mode of operation.

In some embodiments, the processor identifies a default mode ofoperation. In some embodiments, the processor identifies the defaultmode of operation to be the touchpad mode after an auto mode selectionhas occurred followed within a predefined amount of time by adetermination of a sliding motion at least on or near the multi-modearea based on the at least one sense signal. In some embodiments, theprocessor identifies the default mode to be the numeric keypad mode ifafter the auto mode selection no sliding motion is detected within thepredefined amount of time based on the at least one sense signal.

In some embodiments, the processor determines mode of operation to bethe touchpad mode, if the processor detects a touch-and-slide motion atthe multi-mode area based on the generated at least one sense signal andthe motion signal. In some embodiments, the processor determines mode ofoperation to be at least one of the numeric keypad mode or the keyboardmode, if the processor detects only a tap motion based on the generatedmotion signals and the detected tap motion did not occur within athreshold amount of time since the detected touch-and-slide motion.

In some embodiments, the processor returns interpretation of thegenerated at least one sense signal and the motion signal associatedwith the multi-mode area to the default mode after a predefined periodof time has expired since a previously generated at least one sensesignal and motion signal associated with the multi-mode area.

In some embodiments, the surface includes a display device coupled tothe processor and the processor is configured to generate an image andpresent the generated image in the multi-mode area of the surface,wherein the generated image is associated with current mode ofoperation.

In some embodiments, the surface includes a static representation of atleast one of a numeric keypad, keyboard or touchpad.

The embodiments disclosed herein also provide systems and methods thatallow the user to rest their fingers on the keys of an onscreen keyboarddisplayed on a touch-sensitive screen and dynamically define thelocation, orientation, shape, and size of the onscreen keyboard. Ratherthan the user having to take care to place their fingers on the keys(which typically would require tactile markers on said keys), the systemdynamically moves the location of the onscreen keyboard to where theuser's fingers are already resting.

In one aspect, the process defines a “home-row definition event,” whichis an action performed by the user that causes the system to redefinewhere the home-row of the onscreen keyboard is located. This location isdynamically established based on the user's action.

In another aspect, the home-row definition event is defined as the userresting all four fingers of both hands simultaneously on thetouch-sensitive surface for a preset period of time (e.g., 1 second).

In still another aspect, the home-row definition event is defined as theuser double-tapping all four fingers of both hands on thetouch-sensitive surface and then resting them on the surface after asecond tap.

In yet another aspect, the home-row definition event is defined as theuser resting all four fingers of both hands simultaneously on thetouch-sensitive surface and then pressing them down momentarily.

These actions (as well as others) are initiated by the user to indicateto the system that the user's fingers are in the home-row restingposition. The system then orients the onscreen keyboard accordingly.Note that the keys on the home-row needn't be in a continuous line (asthey are on most electromechanical keyboards). Rather, the location ofeach key on the home-row is defined by the placement of the user's eightfingers during a home-row definition event as sensed by touch sensors,and then extrapolated for keys that are not “home-row resting keys.” Inthis way the home-row could be along two separate lines, one for eachhand placement, or may even form two curves.

Once a home-row definition event has taken place, the system providesfeedback to the user in numerous ways. In one aspect, the systemprovides visual feedback by causing the onscreen keyboard to appearbeneath the user's fingers. In another aspect, the system provides anaudible cue. In yet another aspect, the system causes the touch-screento momentarily vibrate.

In one aspect, according to the user's preference, the onscreen keyboardremains visible continuously while typing is taking place.Alternatively, the onscreen keyboard becomes transparent after thehome-row definition event. In another aspect, the onscreen keyboardbecomes semitransparent, allowing the user to see through the keyboardto content on the screen below.

In yet another aspect, the onscreen keyboard cycles between visible andinvisible as the user types. Each time the user taps on the “hidden”onscreen keyboard, the onscreen keyboard temporarily appears and thenfades away after a user-settable amount of time.

In yet another aspect, only certain keys become visible after eachkeystroke. The keys which become temporarily visible are those keys thatare most likely to follow the immediately preceding text input sequence(as determined by word and letter databases stored in the system).

In yet another aspect, the onscreen keyboard becomes temporarily visiblewhen the user, with fingers resting in the home-row position, pressesdown on the surface with their resting fingers.

In still yet another aspect, the onscreen keyboard becomes visible whenthe user performs a predefined action on the edge of the enclosureoutside of the touch sensor area, such as a double- or triple-tap.

In one aspect, the home-row resting keys are defined as the eight keysrested upon by the four fingers of each hand. In yet another aspect, theresting keys may be fewer than eight keys to accommodate users who maynot have use of all eight fingers.

In another aspect, the system disambiguates which key was intendedaccording to movement of a particular finger in an intended direction.For example, the user lifts their ring finger and moves it slightlydownward and taps. The user may not have moved far enough to reach thevirtual location of the adjacent key, but their intention was clearly toselect it since they moved from their resting position by a definablethreshold distance and tapped in the direction of the adjacent key. Eventhough the tap may not have occurred on the adjacent key in thisexample, the system will select it.

In another aspect, the system adjusts the probability of each key beingselected, based on the text sequence that immediately preceded it. Thisprobability is used in conjunction with the tap location algorithmdescribed in the previous paragraphs to determine the most likely keythe user intended to tap on.

In yet another aspect, the system automatically accounts for “userdrift” as they type on the onscreen keyboard. Without the benefit oftactile feel for each key, it is easy for the user to move their handsslightly as they type. The system tracks this behavior by comparing thecenter of the intended key with the actual location that the usertapped. If a consistent drift is detected over the space of consecutivekey events, the system shifts the location of the keys accordingly toaccommodate the drift. Again, rather than make the user take care wherethe keys are, the system moves the keys to where the user's fingers arealready located.

If the user drifts too far to the point of straying off of thetouch-sensitive area, the system warns them with an audible, visual,and/or vibrating cue.

In another aspect, the method and system monitor for user taps that areon the surface of the portable computing device, but not within theboundaries of the touch sensor. For example, the user may tap an edge ofthe device's enclosure to indicate a spacebar actuation. As with othertap events, the system correlates the signals from the touch sensors andvibration sensors to determine the tap location. When an absence ofsignal is detected by the touch sensor, the system recognizes the eventas an “external tap” (i.e., a tap on the surface of the device, butoutside the boundaries of the touch sensors). External taps generateunique vibration waveforms depending on their location on the enclosure.Characteristics of these waveforms are stored in a database and are usedto uniquely identify the general location of the external tap. Theexternal taps, once identified, can be assigned to keyboard functions(such as space or backspace).

In some embodiments, a device including a display, a plurality of touchsensors coupled to the display, a plurality of motion sensors, and aprocessor in signal communication with the display, the plurality oftouch sensors, and the plurality of motion sensors is provided. In someembodiments, the plurality of touch sensors are configured to generatesense signals based on sensed user contact with the display. In someembodiments, the plurality of motion sensors are configured to generatemotion signals based on sensed vibrations of a housing. In someembodiments, the processor is configured to generate and present on thedisplay an image of a keyboard having a plurality of keys based on atleast one of the generated sense signals or the generated motionsignals. In some embodiments, the housing is configured to contain thedisplay, the plurality of touch sensors, the plurality of motionsensors, and the processor.

In some embodiments, the processor is configured to determine locationof the keyboard image on the display based on the generated sensesignals. In some embodiments, the processor is configured to determinelocation of the keyboard image on the display based on determination ofexistence of a home-row definition event. In some embodiments, theprocessor determines an existence of the home-row definition event whentwo or more generated sense signals are determined to be active for apredefined amount time.

In some embodiments, the processor is configured to: 1) determinelocations of home-row keys of the keyboard image based on determinationof location of the generated two or more sense signals; and 2) determinelocations of non-home-row keys of the keyboard image based on determinedlocation of at least one of the home-row keys.

In some embodiments, the processor is configured to: 1) determine sizesof home-row keys of the keyboard image based on determination oflocation of the generated two or more sense signals; and 2) determinesizes of non-home-row keys of the keyboard image based on determinedlocation of at least one of the home-row keys.

In some embodiments, the processor is configured to: 1) determineorientations of home-row keys of the keyboard image based ondetermination of location of the generated two or more sense signals;and 2) determine orientations of non-home-row keys of the keyboard imagebased on determined location of at least one of the home-row keys.

In some embodiments, the housing further includes a vibration deviceconfigured to generate vibrations at one or more frequencies. In suchembodiments, the processor is configured to cause the vibration deviceto activate at a predefined frequency based on the home-row definitionevent.

In some embodiments, the housing includes a vibration device configuredto generate vibrations at one or more frequencies. In such embodiments,the processor is configured to: 1) place the presented keyboard in astatic mode of operation; 2) determine location of at least one userfinger based on the sensor signal; and 3) cause the vibration device tocreate a vibration at a predefined frequency when the determinedlocation of the at least one user finger is within a threshold distancefrom the at least one home key.

In some embodiments, the vibration device is configured to alterintensity of the vibration based on distance of the at least one userfinger from the corresponding home key.

In some embodiments, the housing includes an audio device configured togenerate audio signals at one or more frequencies. In such embodiments,the processor is configured to: 1) place the presented keyboard in astatic mode of operation; 2) determine location of at least one userfinger based on the sensor signal; and 3) cause the audio device tocreate an audio signal at a predefined frequency when the determinedlocation of the at least one user finger is within a threshold distancefrom the at least one home key.

In some embodiments, the audio device is configured to alter intensityof the audio signal based on distance of the at least one user fingerfrom the corresponding home key.

In some embodiments, the processor is configured to: 1) periodicallyreceive sense signals associated with continual user finger contact withthe display; 2) determine if the received periodic sense signalsindicate drift from locations of the sense signals used during thegeneration and presentation of the keyboard image; and 3) move at leastone key of the keyboard image on the display based on a drift indicatedof the at least one key.

In some embodiments, the device includes an output device and theprocessor is configured to: 1) determine if the periodically receivedsense signals indicate user finger contact drift is within a thresholddistance of an edge of the display; and 2) output a signal to the outputdevice if user finger contact drift was determined to be within thethreshold distance.

In some embodiments, the processor is configured to: 1) sense a usertyping action based on the generated sense signals and the generatedmotion signals; and 2) change the keyboard image to be at least one ofsemitransparent or invisible when the user typing action is not sensedfor a predefined amount of time.

In some embodiments, after the keyboard image has been made at least oneof semitransparent or invisible, the processor is configured to causethe keyboard image to appear at least less transparent when a usertyping action has been sensed.

In some embodiments, the processor is configured to: 1) determine atleast one next most likely key to be activated based one or moreprevious key activations; and 2) uniquely display the determined atleast one next most likely key.

In some embodiments, the processor is configured to: 1) determinerelative movement of one or more user fingers from the home-row keysbased on the generated sense signals; and 2) generate a key activationevent based on the generated motion signals and the determined relativemovement.

In some embodiments, the processor is configured to: 1) generate one ormore candidate keys based on at least a portion of the generated sensesignals and the generated motion signals; and 2) generate a keyactivation event by disambiguating the generated one or more candidatekeys using a statistical probability model.

In some embodiments, the processor is configured to: 1) determine a sizevalue for at least one key based on statistical probability model and atleast one previous key activation event; and 2) alter the keyboard imagebased on the determined size value for the at least one key.

In some embodiments, the processor is configured to: 1) cause thepresented keyboard image to be invisible in an active state based on asensed first user action; and 2) cause the presented keyboard image tobe invisible in an inactive state based on a sensed second user action.

In some embodiments, the generated at least one motion signal isassociated with a location relative to the housing. In such embodiments,the processor is configured to identify a function based on the locationrelative to the housing, when the at least one motion signal has beengenerated and no sense signals have been generated.

In some embodiments, systems and methods that allow the user to resttheir fingers on a touch-sensitive surface and make selections on thatsurface by pressing are provided. Touch capacitance sensors thattypically provide X and Y location data associated with a user's touchare also used to discern finger pressure in the Z direction. This allowsthe user to make an actuation on the touch screen by simply pressingharder at a location where they may already be resting their finger(s).

In one aspect, the process discerns between the actions of tapping onthe surface, resting on the surface, and pressing on the surface. Itdoes so using, in part, thresholds for the touch signal (which may bedynamically altered to accommodate the touch signatures of differentusers). The process also takes into account the rate of the rising edgeof the touch signal to discern between a tap, a resting action, and apress.

It is desirable to allow a human user to rest their hands and/or fingerson a touch surface without causing an actuation, yet still allow otheractions issued by the user through touch, such as a press, to beinterpreted as commands by the system.

In some embodiments, a system including a touch-sensitive surface and aprocessor is provided. In some embodiments, the touch-sensitive surfaceincludes a plurality of touch capacitive sensors associated withactionable locations on the surface. In some embodiments, the processoris configured to: 1) determine a user interaction with the touchsensitive surface as a resting action based on one or more signalsreceived from one or more of the plurality of touch sensors, wherein thesignals are above a first threshold value; and 2) determine a userinteraction with the touch sensitive surface as a press action based onthe one or more signals received from one or more of the plurality oftouch sensors, wherein the received signals are above a second thresholdvalue.

In some embodiments, the first and second threshold values arepredefined. In other embodiments, the first and second threshold valuesare variable based on individual touch characteristics of each user.

In some embodiments, the processor is configured to assert an activationafter a determined resting action for a particular user interaction isfollowed by a determined press action on an actionable location.

In some embodiments, the processor is further configured to determine apress and hold event when a determined resting action for a particularuser interaction is followed by a determined press action that issustained for longer than a predefined key repeat time.

In some embodiments, the processor is further configured to: 1)determine a user interaction as a selection event based on one of thesignals having a leading rising edge with a rate-of-change that exceedsa first rate-of-change threshold followed within a predefined amount oftime by the signal decreasing in value at a rate-of-change greater thana second rate-of-change threshold. In some embodiments, the first andsecond rate thresholds are the same.

In some embodiments, the amplitude of the signal is greater than atleast the first threshold value.

In some embodiments, the processor is further configured to: determinethe initial user interaction with the touch sensitive surface is theresting action is further determined when a rate-of-change of theleading rising edge of the signal is less than the first rate-of-changethreshold and the one or more signals are above the first thresholdvalue.

In some embodiments, the processor is further configured to: determinethe initial user interaction with the touch sensitive surface is thepress action is further determined when the rate-of-change of theleading rising edge of the signal is less than the first rate-of-changethreshold and the one or more signals are above the second thresholdvalue.

In some embodiments, the system includes an output device configured topresent a response corresponding to the press action.

In another aspect, a system including a touch-sensitive surface and aprocessor in signal communication with the touch-sensitive surface isprovided. In some embodiments, the touch-sensitive surface includes aplurality of touch capacitive sensors associated with actionablelocations on the surface and the sensors are configured to generate oneor more signals. In some embodiments, the processor is configured to: 1)determine a user interaction with the touch sensitive surface is aresting action based on one or more signals received from one or more ofthe plurality of touch sensors, wherein the signals are above a firstthreshold value; 2) after determination of the resting action, determinea peak of the one or more signals and determine a difference inamplitude of a location of the one or more signals associated with theresting action and the determined peak; 3) if the determined differenceis greater than a first predefined delta threshold, determine that auser interaction with the touch sensitive surface is a press action; 4)after determination of the press action, determine that a userinteraction with the touch sensitive surface is at least one of: (a) arest and press release action, if the one or more signals are determinedto be at or below the first threshold value; or (b) a press releaseaction, wherein the processor determines an amplitude of a trough of theone or more signals and determines the user interaction is the pressrelease action if the trough amplitude and the determined peak have adifference that is greater than a second predefined delta threshold.

In some embodiments, the first and second predefined delta thresholdsare the same.

In some embodiments, the system further includes an output deviceconfigured to present a response corresponding to the press action.

In yet another aspect, a system including a touch-sensitive surface anda processor is provided. In some embodiments, the touch-sensitivesurface includes a plurality of touch capacitive sensors associated withactionable locations on the surface, the sensors configured to generateone or more signals. In some embodiments, the processor is configuredto: 1) determine the initial user interaction with the touch sensitivesurface is the resting action is further determined when arate-of-change of the leading rising edge of the signal is less than thefirst rate-of-change threshold and the one or more signals are above thefirst threshold value; 2) determine the initial user interaction withthe touch sensitive surface is the press action is further determinedwhen the rate-of-change of the leading rising edge of the signal is lessthan the first rate-of-change threshold and the one or more signals areabove the second threshold value; 3) after determination of the restingaction, determine a peak of the one or more signals and determine adifference in amplitude of a location of the one or more signalsassociated with the resting action and the determined peak; 4) if thedetermined difference is greater than a first predefined deltathreshold, determine that a user interaction with the touch sensitivesurface is a press action; 5) after determination of the press action,determine that a user interaction with the touch sensitive surface is atleast one of: (a) a rest and press release action, if the one or moresignals are determined to be at or below the first threshold value; (b)or a press release action, wherein the processor determines an amplitudeof a trough of the one or more signals and determines the userinteraction is the press release action if the trough amplitude and thedetermined peak have a difference that is greater than a secondpredefined delta threshold.

In some embodiments, the first and second predefined delta thresholdsare the same.

In some embodiments, the system further includes an output deviceconfigured to present a response corresponding to the press action.

BRIEF DESCRIPTION OF THE DRAWINGS

The implementations disclosed herein are illustrated by way of example,and not by way of limitation, in the figures of the accompanyingdrawings. Like reference numerals refer to corresponding partsthroughout the drawings.

FIG. 1 is a block diagram of an exemplary system formed in accordancewith some embodiments.

FIG. 2 is a graphical representation of a state machine, detailing thestates of resting and pressing, in accordance with some embodiments.

FIG. 3 is a data flow diagram of exemplary processes performed by thesystem shown in FIG. 1, in accordance with some embodiments.

FIGS. 4A and B are plots of waveforms representing the touch signalvalue in the time domain for various press actions, in accordance withsome embodiments.

FIG. 5 illustrates the disruption of an electrical field caused by thecapacitance of a lightly-touching finger, in accordance with someembodiments.

FIG. 6 illustrates the disruption of an electrical field caused by thecapacitance of a finger being pressed strongly into the surface, inaccordance with some embodiments.

FIGS. 7A, 7B, and 7C are waveform plots of a tap selection, a rest, anda press action, all in the time domain, in accordance with someembodiments.

FIG. 8 shows an exemplary process performed by the system shown in FIG.1, in accordance with some embodiments.

FIG. 9A is a schematic of a partial view of an exemplary touch-sensitivesurface formed, in accordance with some embodiments.

FIG. 9B is a schematic of a touch-sensitive surface used to illustrateplacement of a user's fingers on a keyboard, in accordance with someembodiments.

FIGS. 10A through 1 OF show a flowchart of exemplary processes performedby the system shown in FIG. 1, in accordance with some embodiments.

FIG. 11A is a schematic view of a tablet device with a flat-surfacedvirtual keyboard, in accordance with some embodiments.

FIGS. 11B and 11C are schematics of a touch-sensitive surface withonscreen keyboard displays, in accordance with some embodiments.

DETAILED DESCRIPTION

Reference will now be made to embodiments, examples of which areillustrated in the accompanying drawings. In the following description,numerous specific details are set forth in order to provide anunderstanding of the various described embodiments. However, it will beapparent to one of ordinary skill in the art that the various describedembodiments may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, circuits, andnetworks have not been described in detail so as not to unnecessarilyobscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are used onlyto distinguish one element from another. For example, a first devicecould be termed a second device, and, similarly, a second device couldbe termed a first device, without departing from the scope of thevarious described embodiments. The first device and the second deviceare both devices, but they are not the same device.

The terminology used in the description of the various embodimentsdescribed herein is for the purpose of describing particular embodimentsonly and is not intended to be limiting. As used in the description ofthe various described embodiments and the appended claims, the singularforms “a,” “an,” and “the” are intended to include the plural forms aswell, unless the context clearly indicates otherwise. It will also beunderstood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“includes,” “including,” “comprises,” and/or “comprising,” when used inthis specification, specify the presence of stated features, integers,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting”or “in accordance with a determination that,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event]” or “in accordance with a determination that [astated condition or event] is detected,” depending on the context. Asused herein, the term “exemplary” is used in the sense of “serving as anexample, instance, or illustration” and not in the sense of“representing the best of its kind.”

FIG. 1 shows a block diagram of an exemplary device 100 (comprisinghuman interface hardware components, memory, and processors) forproviding a touch interface that, in some embodiments, discerns betweentapping, resting, and pressing, provides a multiplexed numeric keypadand touchpad, and/or provides an adaptive onscreen keyboard userinterface for alphanumeric input. The device 100 includes one or moretouch sensors 120 that provide input to a CPU (processor) 110. The touchsensors 120 notify the processor 110 of contact events when a surface istouched. In one embodiment, the touch sensor(s) 120 (e.g., capacitivetouch sensors or a pressure sensor, such as a strain gauge), or theprocessor 110, include a hardware controller that interprets raw signalsproduced by the touch sensor(s) 120 and communicates the information tothe processor 110, using a known communication protocol via an availabledata port. In some embodiments, the processor 110 generates an imagethat is presented on an optional display 130 (touch surface) oralternatively, the display may be static. In some embodiments, theprocessor 110 communicates with an optional hardware controller to causethe display 130 to present an appropriate image.

In some embodiments, the touch sensors 120 include capacitive touchsensors. In some embodiments, the analog values produced by thecapacitive touch sensors are obtained and analyzed (e.g., by processor110 of device 100). When a user presses their finger against the display130 (e.g., a display with a touch-sensitive surface or a touchscreendisplay), more of their finger comes in contact with the surface as theflesh of their fingers flattens against the surface. In someembodiments, this increase in contact causes a corresponding change inthe touch capacitive signal (e.g., the analog value produced by thecapacitive touch sensors of touch sensors 120). In this way, a press isdetected on the touch-sensitive surface.

In some embodiments, the touch sensors 120 include pressure sensors(e.g., a strain gauge) within device 100 or a component thereof (such asan external keyboard). The sensed weight (or pressure) of a contact on atouch-sensitive surface increases as the user presses on a key (e.g., asoft keyboard displayed on display 130). In some implementations, tolocalize the press, more than one strain gauge is used and the resultingsignals detected by each strain gauge are used to determine a locationon the touch-sensitive surface corresponding to the press. In someembodiments, the determined location is correlated with additional touchsensor data (e.g., data obtained by capacitive touch sensors included intouch sensors 130) to further refine the location of the press. In someembodiments, the device 100 includes pressure sensors and capacitivetouch sensors and device 100 detects occurrence of a press based onoutput from the pressure sensors and then determines a location on thetouch-sensitive surface corresponding to the press using the change incapacitance of the capacitive touch sensors.

In some embodiments, touch sensors 120 include resistive sensors thatare used to determine/detect press actions on the touch-sensitivesurface. More specifically, as the user touches the touch-sensitivesurface, the impedance between two planes on the surface changes andthis change is used to detect the different between a rest and a press.

In some embodiments, the device 100 optionally includes one or morevibration sensors 125 (e.g., accelerometers) that communicate signals tothe processor 110 when the surface is tapped, in a manner similar tothat of the touch sensor(s) 120. The processor 110 generates a keyboardimage that is presented on the display 130 (e.g., a display with atouch-sensitive surface) based on the signals received from the sensors(e.g., touch sensors 120 and vibration sensors 125). A speaker 135 isalso optionally coupled to the processor 110 so that any appropriateauditory signals are passed on to the user as guidance (e.g., errorsignals). A vibrator 180 is also optionally coupled to the processor 110to provide appropriate haptic feedback to the user (e.g., errorsignals).

In some embodiments, the processor 110 is in data communication with amemory 140, which includes a combination of temporary and/or permanentstorage, and both read-only and writable memory (random access memory orRAM), read-only memory (ROM), writable nonvolatile memory, such as FLASHmemory, hard drives, floppy disks, and so forth. The memory 140 includesprogram memory 150 that includes all programs and software such as anoperating system 151, press detection software component 152, adaptiveonscreen keyboard (“OSK”) software component 154, User GestureRecognition software 155, and any other application software programs153. The memory 140 also includes data memory 160 that includes SystemSettings 161, a record of user options and preferences 162 (e.g.,required by the User Gesture Recognition software 155), statisticaldatabase(s) 181 (e.g., word database(s)), and any other data 163required by any element of the device 100.

In some embodiments, the device 100 allows the user to perform at leastthree interactions on the touch-sensitive surface of display 130 (alsoreferred to as a touchscreen or a touch surface): a touch-and-releaseselection (or a “tap”), a resting action wherein they rest two or morefingers simultaneously on the touch surface, and a pressing action.Being able to distinguish between these three actions significantlyimproves the flexibility and usefulness of the user interface of thedevice 100. For example, the touch surface can be used as a keyboard,allowing the user to rest their fingers on it as they would whiletouch-typing on a traditional keyboard.

In some embodiments, once a home-row event has been detected by theprocessor 110 based on signals from the sensors (e.g., touch sensors 120and vibration sensors 125), the processor 110 positions a virtualonscreen keyboard beneath the user's fingers on the display 130. As theuser types, the processor 110 constantly monitors the placement of theuser's fingers, as well as tapped locations for each key actuation, andmakes adjustments to the location, orientation, and size of each key(and the overall keyboard) to ensure the onscreen keyboard is locatedwhere the user is typing. In this way, it is possible to account for theuser's “drifting”, or moving their fingers off of the original positionof the onscreen keyboard. If the user drifts too far in one directionso-as to reach the edge of the touch sensor area, the processor 110outputs an audible and/or haptic warning.

At any time, the user may manually re-assign the location of theonscreen keyboard by initiating a home-row definition event (asdescribed above).

In one embodiment, haptic feedback is provided via the vibrator 180 whenthe user positions their index fingers on the keys commonly-referred toas the “home keys” (F and J keys on a typical English keyboard). In oneembodiment, a momentary vibration is issued when the user rests theirfingers on the keys using a slightly different frequency of vibrationfor left and for right. In this manner, the user may choose to movetheir hands back into a fixed home-row position, when the user choosesto manually re-assign the location of the onscreen keyboard byinitiating a home-row definition event (in other words, processor 110does not dynamically change the position of the onscreen keyboard, thelocation of the onscreen keyboard instead changes in response to theuser's initiation of a home-row definition event). In anotherembodiment, the intensity of these vibrations may change depending uponfinger position relative to the home keys of the fixed home-row.

The device 100 allows the user to type without looking at their fingersor the virtual keyboard. It follows, then, that the keyboard need not bevisible at all times. This allows valuable screen space to be used forother purposes.

In one embodiment, the visual appearance of the keyboard varies itsstate between one or more of the following: visible, partially visible,invisible, and semitransparent. The full keyboard visually appears whena home-row definition event takes place or when the user has restedtheir fingers without typing for a settable threshold amount of time. Asthe user begins to type, the keyboard fades away to invisible until theuser performs any one of a number of actions including, but not limitedto: a home-row definition event, pausing typing, pressing on fourfingers simultaneously, or some other uniquely identifying gesture. Inanother embodiment, the keyboard does not fade away to be completelyinvisible, but rather becomes semitransparent so the user can stilldiscern where the keys are, but can also see content on the display thatis “beneath” the onscreen keyboard.

In one embodiment, the keyboard temporarily “lights”, or makes visible,the tapped key as well as those that immediately surround the tapped keyin a semitransparent manner that is proportional to the distance fromthe tapped key. This illuminates the tapped region of the keyboard for ashort period of time.

In one embodiment, the keyboard becomes “partially” visible with thekeys having the highest probability of being selected next lighting upin proportion to that probability. As soon as the user taps on a key,other keys that are likely to follow become visible or semi-visible.Keys that are more likely to be selected are more visible, and viceversa. In this way, the keyboard “lights” the way for the user to themost likely next key(s).

In one embodiment, the onscreen keyboard is made temporarily visible bythe user performing tap gestures (such as a double- or triple-tap inquick succession) on the outer rim of the enclosure surrounding thetouch-sensitive surface.

The various modes of visual representation of the onscreen keyboard maybe selected by the user via a preference setting in a user interfaceprogram (e.g., by modifying a preference setting stored in useroptions/preferences 162).

FIG. 2 is a state diagram that illustrates how a press state isdetermined by the processor 110. The system is initialized in 200 andthen enters the idle state 205 where no touch is detected. When a touchsignal is detected, the system begins to measure the accumulation of thesignal. When the accumulation reaches a pre-defined threshold called theBinary Rest Threshold in 206, the system proceeds to the Plateau State210. In the Plateau State 210, the user is deemed to be resting theirfinger(s) on the touch surface. If the user removes their finger(s) fromthe surface and the Slope Accumulation drops below the Binary RestThreshold in 211 then the system returns to Idle State 205. From thePlateau State 210 a user may press their finger harder into the surfacecausing the Slope Accumulation to continue to increase past apre-defined Positive Press Threshold 212, upon which the system proceedsto the Positive Press Detect State 215 and asserts a press action. Aslong as the user maintains the pressure while in the Positive PressDetect State 215, the system maintains the press assertion (similar toholding down a key on a traditional keyboard). Once in the PositivePress Detect State 215, the user may lift their finger(s) from thesurface causing the Slope Accumulation to decrease below the Binary RestThreshold in 217 and the system returns once again to the Idle State205. However, while in the Positive Press Detect State 215, the user mayreduce the pressure of the pressing action without completely removingtheir finger. In this case, a negative inflection point occurs where thetouch signal decreases to a point and then either levels out or beginsto increase again (i.e. where the slope of the touch signal curve iszero as it passes from negative to positive). When a negative inflectionpoint is detected the system determines if the Slope Accumulation hasdecreased below a Negative Press Threshold point in 216, at which pointthe system advances to the Negative Press Detect State 220 and the pressaction is released. Note that the Negative Press Detect State 220 issimilar to the Plateau State 210 in that the user is deemed to beresting. However, the absolute value of the touch signal may be quitedifferent between the two states. When in the Negative Press DetectState 220 the system watches for a maximum inflection point (where theslope of the curve is zero as it passes from positive to negative). Whena max inflection point takes place and the Slope Accumulation exceedsthe Positive Press Threshold in 221, the system returns to the PositivePress Detect State 215 and asserts a press action. Alternatively, whilein the Negative Press Detect State 220, if the Slope signal falls belowthe Binary Rest Threshold in 222 then the user is deemed to have liftedtheir finger off the surface and the system returns to the Idle State205.

FIG. 3 is a data flow diagram that shows how the CPU 110 measures,stores, and analyzes the touch signal. In block 300 the system acquiresthe raw sensor data from an analog to digital convertor (ADC). Thesignal is then passed through a low-pass filter in block 305 in order tosmooth out any high frequency noise that may be present in the signal.The result is then stored in a Cache (2) in block 310. The slope of thesignal is then analyzed in block 315, followed by detection of theminimum and maximum inflection points of the signal in block 320. Inblock 325 the system accumulates the slope changes and stores the resultin Cache (1) in block 330. This calculation determines the amplitudedifference between the min and max inflection points. In block 335, therate of change of the signal is determined and stored in Cache (1) inblock 340. The rate of change of the signal is helpful in determiningthe difference between a tap selection, a resting set-down action, and apress (as illustrated in FIGS. 7A, 7B, and 7C). In block 345 of FIG. 3,the system determines the current press state.

FIGS. 4A and 4B are representations of the touch signal going through anumber of conditions resulting in press actions being issued by thesystem. In FIG. 4A the system follows a very simple process of usingfixed threshold values to determine the difference between a restingaction and a press. The user touches the surface at 4000 causing thetouch signal to rise above the pre-defined Rest Threshold 4050, at whichpoint the signal levels off at 4010 causing an inflection point andputting the system into the Plateau State 210. Sometime later, the userpresses harder on the surface causing the touch signal to increase abovethe Press Threshold 4055 to a local maxima value at 4020 at which pointthe system asserts a press action (indicated by the black circle). Thesystem continues looking for maxima and minima inflection points. Theinflection points found at 4025 and 4030 are ignored since they occurabove the Press Threshold, meaning the press asserted at 4020 continuesto be asserted. At 4035 the system detects a minima inflection pointthat falls above the Rest Threshold 4050 and below the Press Threshold4055 at which point it asserts a press release action (indicated by thehollow circle). The user then presses again causing the touch signal toincrease past the Press Threshold. The system detects the maximainflection point at 4040 and assets another press action. The user thencompletely lets go, causing the touch signal to fall back to zero.Although no inflection point is detected, at 4045 the system recognizesthat the touch signal has fallen below the Rest Threshold 4050 andassets a press release action.

The method described in the above paragraph associated with respect toFIG. 4A is straight-forward, but fails to discern the possible pressaction that takes place between 4025 and 4030. When a user performsmultiple presses in quick succession, the touch signal often remainsabove the Press Threshold even on the press release action. In order toremedy this short-coming an embodiment is illustrated in FIG. 4B.

Referring to FIG. 4B, the user touches the surface at 4100 causing thetouch signal to rise above a pre-defined Rest Threshold 4150, at whichpoint the signal levels off at 4110 causing an inflection point whichthe system discerns as a Rest assertion and places the state machineinto the Plateau State 210. Sometime later, the user presses harder onthe surface causing the touch signal to increase to a local maximumvalue at 4120. The relative change in the signal from 4110 to 4120 iscompared with another threshold called the Press Assertion DeltaThreshold. If the increase in signal between 4110 and 4120 is greaterthan the Press Assertion Delta Threshold then a press action is assertedby the system at 4120 (indicated by the solid black circle). Followingthis assertion, the user decreases the touch pressure between 4120 and4125 but then once again increases the pressure between 4125 and 4130.At 4125, the system detects a minimum inflection point and measures thechange in the touch signal between 4120 and 4125 which is then comparedwith yet another threshold called the Press Release Delta Threshold. Ifthe absolute value of the decrease in the touch signal between 4120 and4125 is greater than the Press Release Delta Threshold then a releaseaction is asserted by the system (indicated by the hollow circle). Asimilar process takes place between 4130, 4135, and 4140 only withdifferent amplitudes and rate of change in the signal. Finally, the userstops pressing at 4140 but keeps their finger in contact with thesurface in a resting action at 4145, at which point the system asserts apress release action. After some amount of time, the user then removestheir finger from the touch surface and the signal quickly falls tozero. As the signal decreases through the Rest Threshold the systemasserts a Rest release action at 4150.

In one embodiment the two methods described in FIG. 4A and FIG. 4B maybe selectively combined.

FIG. 5 illustrates one of many possible embodiments in how atouch-sensitive surface can be implemented using capacitance. Atouch-sensitive surface 500 is made up of one or more sensors in whichan electrode 510 emits an electrical signal forming an electrical field530, 540, and 570. An adjacent electrode 520 couples with a portion ofthe formed electrical field 570. The coupled signal at the adjacentelectrode 520 is detected and measured by the system. As a human finger550 touches the surface 500, a portion of the electrical field 540couples with the finger, resulting in less of the electrical field 570coupling with the second electrode 520. The processor 110 receives adigital representation of the analog voltage measurement obtained fromthe second electrode 520 then detects the change of the signal at thesecond electrode 520 and determines a touch has taken place. The degreeto which the electrical field 540 couples with the human finger 550 isdependent, in part, on the amount of surface area 560 with which thefinger comes in contact. A “light” touch is shown in FIG. 5 where thefinger 550 is just making contact with the touch surface 500. Arelatively lower amount of the electrical field 540 is disrupted by thelight touch.

FIG. 6 illustrates the effects of a stronger press on the touchcapacitance signals. A touch-sensitive surface 600 is made up of one ormore sensors in which an electrode 610 emits an electrical signalforming an electrical field 630, 640, and 670. An adjacent electrode 620couples with a portion of the formed electrical field 670. The coupledsignal at the adjacent electrode 620 is detected and measured by thesystem. As a human finger 650 presses hard on the surface 600, arelatively larger portion of the electrical field 640 couples with thefinger, resulting in less of the electrical field 670 coupling with thesecond electrode 620. The processor 110 receives a digitalrepresentation of the analog voltage measurement obtained from thesecond electrode 620 then detects the change of the signal at the secondelectrode 620 and determines a press has taken place. The degree towhich the electrical field 640 couples with the human finger 650 isdependent, in part, on the amount of surface area 660 with which thefinger comes in contact. A “heavy” touch, or press, is shown in FIG. 6where the finger 650 makes strong contact with the touch surface 600causing the finger to flatten out at 660. A relatively larger amount ofthe electrical field 640 is disrupted by the pressing action.

FIGS. 7A, 7B, and 7C illustrate the three actions of a tap selection, aresting set-down action, and a set-down press action respectively. Boththe amplitude of the touch signal and the slope of the leading edge ofthe signal are used to determine which action is being initiated by theuser. In FIG. 7A the user quickly taps on a key causing the signal toexceed a pre-defined first threshold indicating a valid touch has takenplace. The rising slope of the signal is steep, as is the falling edge,and it peaks between the First Threshold and the Second Threshold (theconditions for a “tap” selection). FIG. 7B illustrates the signal thatmeets the conditions for a resting set-down action. In this case, therising edge of the touch signal is relatively slow (as compared to a tapsignal) and the amplitude of the signal stabilizes between the First andSecond Thresholds. FIG. 7C illustrates the signal that meets theconditions for a set-down press action. In this case, the rising edge ofthe touch signal is relatively slow as compared to the tap signal, butsimilar in slope to the rising edge of a rest set-down action. However,the amplitude of the signal continues beyond the Second Thresholdindicating the user has pressed harder than a normal touch. The slowerrise time, but higher amplitude indicates a set-down pressing action hastaken place.

Being able to distinguish between a tap selection, a set-down restingaction, and a pressing action is critical in allowing the user to resttheir fingers on a touch surface. Further, using the same sensors todetect all three actions has the advantages of keeping the cost of thesystem relatively lower and simpler.

FIG. 8 shows a flow chart of an exemplary process 800 that allows thesame physical area on a touchscreen keyboard to be used to perform thefunctions of both a numeric keypad and touchpad. The process 800 is notintended to fully detail all the software in its entirety, but isprovided as an overview and an enabling disclosure.

The process 800 is provided by the User Gesture Recognition Software155. At block 805, when the process is first started, various systemvariables are initialized. For example, event time out (threshold time)is set to zero. At block 810, the process waits to be notified that usercontact has occurred within the common area. While the system is waitingin block 810, a counter is incremented with the passage of time. Onceuser contact has occurred, block 815 determines if the counter hasexceeded the maximum time (threshold) allowed for user input (stored asa user option in Data Memory 160).

If the maximum time allowed for user input has been exceeded, then thesystem resets the mode of the common area to the default mode in block820. At a decision block 825, the processor 110 determines whether ornot the current mode is in touchpad mode. If the current mode is in thetouchpad mode, the processor 110 interprets the user contact as atouchpad event and outputs the command accordingly in block 830.

If the current mode is not in the touchpad mode, then the processor 110assumes the common area is in number pad (numpad) mode and proceeds todecision block 835. In touchpad operation, the user will make an initialtouch followed by a sliding motion with their finger (or multiplefingers). In numpad operation, the user will tap on a number key andtypically will not slide their finger. The processor 110 uses thisdifference in typical operation to interpret the user's input indecision block 835 and if a touch-and-slide motion is detected by theprocessor 110 based on signals provided by the sensors (e.g., touchsensors 120 and vibration sensors 125), the processor 110 changes thecurrent mode to the touchpad mode in block 840, and outputs the useraction as a touchpad event in block 845. If the user action is not atouch-and-slide motion then the user action is output by the processor110 as a numpad event in block 850. After blocks 830, 845, 850, theprocess 800 returns to block 810.

Note that single taps (or multiple taps in succession) are also commonwhen using a touchpad, and are commonly assigned to functions such as“select” or what is commonly referred to as a “mouse left button”action. These types of actions typically occur shortly after atouch-and-slide motion, and so the system will still be in touchpad mode(since the counter will not yet have reached the threshold in block815).

Other user gestures on the touchpad are interpreted and assigned tofunctions, such as multiple finger swipes across the touchpad. While thedevice 100 is in the touchpad mode, all these gestures are interpretedas touchpad input and sent to the device's operating system as such tobe interpreted by whatever system software resides therein. In this way,the system and method acts exactly like any other touchpad when intouchpad mode.

In one embodiment, the default mode is set by the user (typicallythrough control panel software). If the device 100 is at rest with nouser input for the user-settable amount of time (threshold), the mode isrestored to the default mode.

Gesture Recognition

Auto-detect mode includes, but is not limited to, recognition of thefollowing gestures and their assigned actions:

Gesture Description Action Single-touch slide The user slides one fingerMouse cursor movement across the surface in touchpad mode. Single-touchtap immediately The user taps their finger on Left mouse button clickfollowing a single-touch slide the surface within a short period of timeafter sliding their finger across the surface Single-touch double-tapThe user taps their finger twice Left mouse double-click. in quicksuccession on the surface within a short period of time after slidingtheir finger across the surface Dual-touch tap immediately The user tapstwo fingers on Right mouse button click following a single-touch slidethe surface within a short (first threshold) period of time aftersliding their finger across the surface Single-touch tap The user tapsthe surface in a Numpad key press single location after not havingtouched it for a long (second threshold) period of time Single-touch tapfollowing The user taps the surface in a Numpad key press anothersingle-touch tap single location within a short period of time afteralso tapping a single location Single-touch tap followed by a The usertaps the surface in a Left mouse button press and single-touch presssingle location immediately drag followed by a touch and hold on thesurface, then drags their finger across the surface

SWYPE Integration

In another embodiment, the touch surface is used in a fourth mode:SWYPE® keyboard. In this mode, the surface represents a keyboard, onwhich the user may slide their finger from letter to letter,implementing the SWYPE paradigm. This mode is manually selected by theuser through some scheme implemented on the keyboard or computersoftware, or it is selected by functionality provided by the auto-detectmode. In the auto-detect mode, the system observes a sliding motionacross the surface and initially interprets it as touchpad movement.However, if the pattern traces out a legitimate word in the fourth mode,the system intelligently switches into the fourth mode and outputs thetext. The system stays in the fourth mode for as long as the user istyping. To exit the fourth mode and return to touchpad mode, the userperforms a gesture—such as pressing and holding their finger for a fewseconds in the same location. Other gestures could also be recognized.

Keyboard Integration

In another aspect of the system, the touch surface is used in a fifthmode: regular keyboard. In the fifth mode, the surface is reconfiguredto be a standard QWERTY keyboard. Using patent-pending touch-taptechnology, the user can rest their fingers on the touch-sensitivesurface and select keys by “tapping” on them. Because this requires morespace than any of the other paradigms listed above, it is possible thatthe device could be used with a diagonal orientation for the fingers. Inother words, fingers are displaced along the axis connection oppositecorners of the surface. Then, the relative displacement of the fingersfrom this resting position can be detected to determine which keys wereselected (as shown in FIG. 9B). In other words, full sized keys wouldnot be necessary, thus saving space yet allowing comfortable typing.

FIG. 9A shows is a schematic of a keyboard used to illustrate a touch-and tap-sensitive keyboard 900 that incorporates on its forward-facingsurface an area 910 incorporating the functions of both a numeric keypadand touchpad. The term “keyboard” in this application refers to anykeyboard that is implemented on a touch- and tap-sensitive surface,including a keyboard presented on a touch-sensitive display. Thekeyboard 900 includes the outline of the area 910 incorporating thefunctions of the touchpad, the keys assigned to the numeric keypad, aswell as the selection keys commonly referred to as the “left and rightmouse buttons” 930. “Mode” refers to the type of function that isassigned to the commonly-shared area 910. A separate mode key 920 allowsthe user to manually select between Touchpad mode, numeric keypad (or“numpad”) mode, or “Auto” mode (whereby the function assigned to commonarea 910 is determined by the system according to the actions of theuser on the surface of the common area 910).

In one embodiment, the system displays the current mode (touchpad ornumber pad) with visual indicators 920 along with an “Auto” mode visualindicator. In this way, the user can know which mode the system is in atall times. In one embodiment, a mode key 924 is provided below theindicators 920 on the keyboard. User activation of the mode key 924causes the processor 110 to switch to another mode.

In one embodiment, the user may define the default mode to be thetouchpad mode by first selecting Auto mode with the mode key 924immediately followed by a touch-and-slide motion on the common area 910.In the absence of a touch-and-slide motion immediately following theselection of Auto mode, the processor 110 will set the default mode tonumpad mode.

In another embodiment, the touch surface is used in a fourth mode:keyboard. In the fourth mode, the surface represents a keyboard, onwhich the user may enter text using a plethora of methods designed forsmaller touch surfaces (such as those invented for smartphones). Thismode is manually selected by the user through some scheme implemented onthe keyboard or computer software, or it is selected by functionalityprovided by the auto-detect mode. The device stays in keyboard mode foras long as the user is typing. To exit the keyboard mode and return tothe touchpad mode, the user performs a predefined gesture such aspressing and holding all their fingers for a few seconds in the samelocation. The processor recognizes the unique gesture, then changes modeaccordingly. Other gestures could also be recognized.

In another embodiment, the touch surface incorporates a dynamic display.The display changes in accordance with the current mode setting todisplay the appropriate image in the common area. For example, whennumpad mode is selected, a numeric keypad is displayed; when touchpad isselected, a blank rounded rectangle is displayed; and so on.

FIGS. 10A-10F show an exemplary process performed by the device 100. Theflowcharts shown in FIGS. 10A-10F are not intended to fully detail thesoftware in its entirety, but are used for illustrative purposes.

FIG. 10A shows a process 1000 executed by the processor 100 based oninstructions provided by the OSK software component 154. At block 1006,when the process 1000 is first started, various system variables areinitialized, such as minimum rest time, number of finger touchthreshold, drift distance threshold, and key threshold. At block 1008,the process 1000 waits to be notified that a contact has occurred withinthe area of a touch-screen. Then, at block 1010, home-row detectionoccurs based on signals from one or more of the sensors (e.g., touchsensors 120 and vibration sensors 125). Home-row detection is describedin more detail in FIG. 10B. At a block 1012, locations of keys for theto-be-displayed virtual keyboard are determined based on the sensorsignals. The key location determination is described in more detail inFIG. 10C. Next, at block 1016, key activations are processed (see FIGS.10D and 10E for more detail.) At a block 1018, user's finger drift isdetected based on the sensor signals. Finger drift is described in moredetail in FIG. 10F. Then, at block 1020, a virtual keyboard is presentedon the display 130 based on at least one of the determinations made atblocks 1010-1018. The process 1000 repeats when a user removes theireight fingers and then makes contact with the touchscreen.

FIG. 10B shows the home-row detection process 1010. At a decision block1034, the process 1010 determines if a user has rested their fingers onthe touch-screen for a minimum amount of time (i.e., minimum restthreshold). At a decision block 1036, the process 1010 determines if theappropriate number of fingers have rested on the touch surface, thusinitiating a home-row definition event. If the conditions in either ofblocks 1034 or 1036 are not met, the process 1010 exits without changingthe location of the onscreen keyboard.

Once both the time and number of resting fingers requirements are met,the processor 110 determines the location of the resting fingers, seeblock 1040. A KeySpaceIndex (or “KSI”) value is then determined in block1042. The KSI is used to customize the onscreen keyboard to the size andspacing of the user's fingers.

The KSI may change from one home-row definition event to the next, evenfor the same user. In one embodiment, all four fingers of each hand areresting on the touch surface to initiate the home-row definition event.In such a case, the KSI is given by the following formula:

KSI=(Average RestingKey Spacing)/(Modeled NominalSpacing)=[(a+b+c)/3]/A=(a+b+c)/3A

where,

-   -   A=a modeled nominal distance between keys (typically 19 mm)    -   a=the measured distance between RestingKey1 and RestingKey2    -   b=distance between RestingKey2 and RestingKey3    -   c=distance between RestingKey3 and RestingKey4.

The KSI formula can be adjusted accordingly if fewer than four restingfingers are used to initiate a home-row definition event (as defined ina set of user preferences stored in a database). The KSI is used insubsequent processes.

A data model for a standard onscreen keyboard is stored in memory of thesystem. In this data model, the onscreen keyboard layout is divided intotwo sections: keys normally typed with the right hand, and keys normallytyped with the left hand. Further, each key is related to the home-rowresting key that is rested upon by the finger that is most likely totype that particular key (defined as the “related resting key”). Thelocation of each key is defined in the data model as a relativemeasurement from its related resting key.

An exemplary formula for determining the location of each key is givenas:

Key(x′,y′)=KeyModel(x*KSI,y*KSI)

Where,

-   -   x=the nominal stored x distance from the center of the Related        Resting Key (RRK)    -   y=the nominal stored y distance from the center of the RRK

It is possible that the modified key positions of two or more keys mayoverlap. If that is the case, the size of the overlapping keys isreduced until the overlap is eliminated.

The orientation of the X-Y axis is determined separately for eachresting key. For each of the left and right sectors, a curve is fit tothe resting keys in that sector. The X-Y axis for each key is thenoriented to be the tangent (for the x-axis) and orthogonal-tangent (forthe y-axis) to the curve at the center of that key.

FIG. 10C shows the assigning key locations process 1012. The process1012 is repeated for each key of the keyboard. At block 1052, apre-stored location for each key is retrieved from the database 181,relative to its associated resting key position in the form [RestingKey,Ax, Ay]. For example, the key representing the letter “R” is associatedwith the resting key L1 (typically the letter “F”), and is positioned upand to the left of L 1. Thus, its data set would be [L1, −5, 19] (asmeasured in millimeters). Similar data is retrieved for each key fromthe database 181. At block 1054, a new relative offset is calculated foreach key by multiplying the offset retrieved from the database by theKSI. At block 1058, the absolute coordinates of each key is thendetermined by adding the new offset to the absolute location of theassociated resting key as determined at block 1054. At decision block1060, the process 1012 tests to see if any keys are overlapping, and ifso, their size and location are adjusted at block 1062 to eliminate anyoverlap. Then the process 1012 returns to the process 1000.

FIG. 10D shows the process-key actuations process 1016, whereby theactual key events are determined and output. The process 1016 begins atdecision block 1070 that tests if a valid touch-tap event has occurred.This is determined through a correlation between the touch sensor(s) 120and vibration sensor(s) 125. Candidate keys are scored by applying a keyscoring algorithm at block 1072. The key with the highest score is thenoutput at block 1074 and the process 1016 returns.

FIG. 10E shows a process for the key scoring algorithm from block 1072of FIG. 10D. At block 1080, signals received by the touch sensors 120and the vibration sensors 125 are correlated to determine where theuser's tap took place and to define keys in the immediate vicinity as“candidate keys”. By considering keys surrounding the area of the tap(rather than just the key where the tap took place), the processor 110accounts for ambiguity in the user's typing style. At a decision block1082, the process 1072 tests to see if the user moved their finger froma resting key to type. Note that in typical typing styles, even a10-finger touch typist will not constantly rest all four fingers at alltimes. So, it isn't a prerequisite that a change in a resting key takeplace in order for a valid key to be typed on. However, if a change doestake place to the state of a resting key in the vicinity of thecandidate keys (or if it is a candidate key itself), useful informationcan be obtained from such change as explained at block 1084. At block1084 a virtual line is calculated between the resting key in thevicinity of the tap for which a state change was detected, and thelocation of the tap, as calculated at block 1080. The virtual lineextends beyond the tap location. At block 1084 keys that the projectedline passes through or by are determined and the processor 110 increasesthe score of those keys accordingly. In this way, relative movements inthe direction of the desired key are correlated to that key, even if thetap location doesn't occur directly on the key. At block 1088, theprocessor 110 takes into account the preceding words and characters thatwere typed as compared with linguistic data stored in data memory 181(e.g., statistical databases 181). This includes commonly knowndisambiguation methods such as: letter-pair statistical frequencies,partial-match prediction, inter-word prediction, and intra-wordprediction. Appropriate scoring is assigned to each candidate key. Atblock 1090, the candidate key with the highest score representing thehighest calculated probability of the user's intended selection isdetermined and the process 1072 returns.

FIG. 10F shows the drift detection process 1018 for accommodating whenthe user inadvertently moves their hands (or “drifting”) as they type.The process 1018, at block 1091, compares the actual tap location withthe current center of the displayed intended key, and stores thedifference in the X and Y coordinates ΔX and ΔY. These differences areadded to a previous cumulative total from previous keystrokes at block1092. At decision block 1093, the processor 110 tests if the cumulativedifference in either direction exceeds a pre-stored variable called“DriftThreshold” (as defined from user preference or default data storedin data memory 162 (e.g., user options/preferences 162)). If thethreshold is exceeded, the processor 110 moves the location of theentire keyboard in block 1094 by the average of all ΔXs and all ΔYssince the last location definition event. If the cumulative differencesdo not exceed the DriftThreshold for the entire keyboard, then a similarcalculation for the individual selected key is performed at block 1097.At decision block 1098, the processor 110 tests if the cumulativedifferences for that individual key exceeds the user-defined keythreshold after block 1097 and, if so, adjusts its location at block1099. The key threshold is the permissible amount of error in thelocation of the tap as compared to the current location of theassociated key. When key threshold has been exceeded, the associated keywill be moved. After block 1094, if the decision at block 1098 is No, orafter block 1099, then at block 1095, the processor 110 tests if any ofthe new positions overlap with any other keys and if the overallkeyboard is still within the boundaries of the touch sensors. If thereare any conflicts for either test, they are corrected with a “best fit”algorithm in block 1096 and then exits. Also, if no conflicts are found,the process 1018 returns.

Even though the method will allow the user to type without the onscreenkeyboard being visible, there are still times when a user will want toview the keys. For example, if they don't know which key is associatedwith a desired character, or where certain characters are located on aseparate numeric and/or symbols layer. Other users may not be able totype from rote, knowing by memory where each character is located. Forthese, and other reasons, it is important to visually present theonscreen keyboard on the screen of the device.

According to stored user's preference, the onscreen keyboard can remainvisible continuously while typing is taking place. Alternatively, theonscreen keyboard becomes transparent after the home-row definitionevent. In one embodiment, the onscreen keyboard becomes semitransparentso-as to allow the user to see through the keyboard to content on thescreen below.

In the case where the keyboard is set to be invisible, other content maybe displayed on the full screen. There may be other user interfaceelements, such as buttons, that will appear to be active yet be locatedbelow the invisible onscreen keyboard. In such a case, the device 100intercepts the user's input directed toward such an element and causesthe onscreen keyboard to become visible, reminding the user that it isindeed present. The user may then elect to “put away” the keyboard bypressing a corresponding key on the keyboard. Note that putting away thekeyboard is not the same as making it invisible. Putting away thekeyboard means to “minimize” it off the screen altogether, as is acommon practice on touchscreen devices.

In one embodiment, the onscreen keyboard cycles between visible andinvisible as the user types. Each time the user taps on the “hidden”onscreen keyboard, the onscreen keyboard temporarily appears and thenfades away after a user-settable amount of time.

In one embodiment, only certain keys become visible after eachkeystroke. The keys that become temporarily visible are those keys thatare most likely to follow the immediately preceding text input sequence(as determined based on word and letter databases stored in the system).

In one embodiment, the onscreen keyboard becomes temporarily visiblewhen the user, with fingers resting in the home-row position, pressesdown on the surface with their resting fingers based on changes sensedby the touch sensors 120.

In one embodiment, the onscreen keyboard becomes visible when the userperforms a predefined action on the edge of the enclosure outside of thetouch sensor area, such as a double- or triple-tap.

The onscreen keyboard, if set to appear, will typically do so when atext-insertion condition exists (as indicated by the operating system151), commonly represented visually by an insertion carat (or similarindicator).

In one embodiment, the tactile markers commonly used on the F and Jhome-row keys are simulated by providing haptic feedback (such as avibration induced on the touchscreen) when the user positions theirfingers to rest on those keys. In this way, the user may choose for thekeyboard to remain stationary in the same onscreen position, yet findthe correct placement of their hands by touch only (without looking).

To increase the accuracy of the keyboard, statistical models of languageare used. If a touch/tap event yields an ambiguous key choice, thestatistical models are called upon by the processor 110 to offer the keythat is most likely what the user intended.

This “disambiguation” is different from other methods used for othertext input systems because a permanent decision about the desired keymust be made on the fly. There is no end-of-word delineation from whichword choices can be displayed to the user and the output modified.Instead, each time the user taps on a key, a decision must be made and akey actuation must be sent to a target application program (i.e., textentry program).

Several statistical analysis methods can be employed: partial-matchletter prediction, current-word prediction, next-word prediction, andconjunctive next-word prediction. These are explained in detail in thefollowing sections.

Prediction by Partial Match

A well-known algorithm originally invented for data compression usefulin this case is prediction by partial match (or PPM). Applied to akeyboard, the PPM algorithm is used to predict the most likely nextcharacter, given a string of characters that has already occurred (oflength k). Computing time and resources grow exponential with the valueof k. Therefore, it is best to use the lowest value of k that stillyields acceptable disambiguation results.

By way of example, let k=2. A process looks back at the past twocharacters that have been entered and then compare probabilities from adatabase of the most likely next character(s) to be typed. For example,the underlined letters below show what is used to predict the next mostlikely letter:

An

An

An e

An ex

An exa

An exam

An examp

An exampl

An example

The data storage required for this algorithm for a total number ofpossible keys A is: _(A) ^(k+1)

For a typical onscreen keyboard, this process consumes less than 1 MB ofdata.

The statistical model is built up for each language (although with asmall value for k); the table may be similar for languages with commonroots. The model also dynamically updates as the user enters text. Inthis way, the system learns the users typing patterns and moreaccurately predicts them as time goes on.

Language variants are provided in the form of language-specificdictionaries configured through an operating system control panel. Thecontrol panel identifies the current user's language from the systemlocale and selects the appropriate prediction dictionary. The dictionaryis queried using a continuously running “systray” application that alsoprovides new word identification and common word usage scoring.

In one embodiment, a database made up of commonly used words in alanguage is used to disambiguate intended key actuations. The algorithmsimply compares the letters typed thus far with a word database, andthen predicts the most likely next letter based on matches in thedatabase.

For example, say the user has typed “Hel.” Possible matches in the worddatabase are:

Hello (50)

Help (20)

Hell (15)

Helicopter (10)

Hellacious (5)

The numbers beside each word represent their “frequency” of use,normalized to 100. (For convenience sake, the total frequencies in thisexample add up to 100, but that would not normally be the case).

The candidate letters that most likely follow “Hel” are:

L (70)—probabilities added for the words “Hello”, “Hell”, and“Hellacious”

P (20)

I (20)

This example is particularly useful, in that the letters L, P, and I areall in close proximity to one another. It is possible, and even likely,that the user may tap on a location that is ambiguously near severalkeys (I, 0, P, or L, for example). By adding word prediction, the choiceis significantly disambiguated; in this example, the obvious most-likelynext letter is “L.”

Note that this implementation of the word prediction algorithm isdifferent from that traditionally used for onscreen keyboards, becauseit is not truly a word prediction system at all: it is a letterprediction system that uses a word database.

In one embodiment, word pairs are used to further disambiguate the mostlikely selected key. With simple word prediction, there is no context todisambiguate the first letter of the current word; it is completelyambiguous. (This disambiguation is reduced slightly for the secondletter of the word, and so on for the remainder of the word.) Theambiguous nature of the first few letters of a word can be significantlyreduced by taking into account the word that was entered immediatelyprevious to the current word; this is called “next-word prediction”.

For example, if the word just typed was “Cleankeys”, common next wordsstored in the database may be:

Keyboard (80)

Inc. (20)

Is (20)

Will (15)

Makes (10)

Touch (5)

If the user ambiguously taps between the I key and the K key for thestart of the next word, the next-word prediction algorithm can helpdisambiguate (in this case, “K” would win).

Logic may dictate that the concept of considering the previous wordtyped could be carried to the previous k words typed. For example, fork=2, the system could store a database that has 2nd-degree next-words(or next-next-words) for every word in the database. In other words,look back at the two previous words in combination to determine the mostlikely word to follow. However, this quickly becomes unwieldy, both interms of space and computing power. It simply isn't practical to storethat many combinations, nor is it very useful, because most of thosecombinations would never occur.

There is, however, a significant exception that is worth considering:words that have a very large number of next-word candidates. Such is thecase for parts of speech known as conjunctions and articles.

The seven most-used conjunctions in the English language are: and, but,or, for, yet, so, nor.

The articles in the English language are: the, a, an.

By special-casing these 10 words, the system improves first-letterpredictions.

Consider the phrase: kick the ______.

Because every noun in the database is most likely a next-word candidatefor the article “the”, there is very little use derived from thenext-word prediction algorithm. However, if the context of “kick” beforethe article “the” is retained, a much richer next-next-word choice isattained. Effectively, a new “word” is stored in the database called“kick the.” This new entity has the following next-word candidates:

Ball (50)

Bucket (20)

Habit (15)

Can (10)

Tires (5)

Thus one can confidently predict that the most likely next letter tofollow the phrase “kick the” is the letter “B.”

Any word that is found combined with a conjunction or article iscombined with those parts of speech to form a new word entity.

A notable difference between the letter-by-letter prediction systemdescribed herein and a word-based prediction system is the ability todynamically reorient the prediction for each letter. For example, if aguess is wrong for a specific key and the desired word subsequentlybecomes clear, the algorithm abandons the choice it made for theincorrect letter and applies predictions for the remaining letters,based on the newly determined target word.

For example:

Ambiguous Text Entered Candidate Keys Predicted Words Predicted LetterKick the B, h, g Ball, bucket, habit, B goat, garage Kick the b A, q, sBall, habit, wage A Kick the ba B, v, space habit B Kick the bab I, k, ohabit I Kick the babi T, r habit T Kick the babit Space, n, m habitspace

As the word progresses, it is shown that the initial letter “B” shouldhave been an “H” (these letters are near one another on the qwertykeyboard layout and one could easily be mistaken for the other). Butrather than commit completely to that first letter, and only considerwords that start with “B,” other candidates are still considered by thesystem in predicting the second letter. So, B, H, and G are consideredas the first letter for subsequent keys. In this way, the mistake isn'tpropagated and the user would need to only make one correction insteadof potentially many.

So, for each new key entered, keys that are adjacent to the new key, aswell as other ambiguous candidates, are considered as possibilities indetermining subsequent letters.

When a mistake is made and the user backspaces and corrects it, thesystem can feed that data back into the algorithm and make adjustmentsaccordingly.

For example, the user ambiguously enters a key in the middle of thekeyboard and the scoring algorithm indicates that potential candidatesare “H,” “J,” and “N”; the scores for those three letter fall into theacceptable range and the best score is taken. In this example, let's saythe algorithm returns the letter “J” as the most likely candidate and sothat is what the keyboard outputs. Immediately following this, the userunambiguously types a <backspace> and then an “H,” thus correcting theerror.

This information is fed back into the scoring algorithm, which looks atwhich sub-algorithms scored an “H” higher than “J” when the ambiguouskey was originally entered. The weighting for those algorithms isincreased so if the same ambiguous input were to happen again, theletter “H” would be chosen. In this way, a feedback loop is providedbased directly on user corrections.

Of course, the user can make typing mistakes themselves that are not theresult of the algorithm; it correctly output what the user typed. So,care must be taken when determining if the user correction feedback loopshould be initiated. It typically occurs only when the key in questionwas ambiguous.

A user-settable option could allow the keyboard to issue backspaces andnew letters to correct a word that was obviously wrong. In the exampleabove, once the predictor determines that the only logical word choiceis “habit,” the keyboard would issue backspaces, change the “b” to an“h,” reissue the subsequent letters (and possibly even complete theword).

With so many factors lending to the disambiguation of a key, allalgorithms can potentially add to the candidacy of a key. This approachis called scoring; all algorithms are weighted and then added together.The weighting is dynamically changed, to tune the scoring algorithm tothe user's typing style and environment.

FIG. 11A shows a schematic view representative of a typical handheldtablet computer 1150 that incorporates on its forward-facing surface atouch-sensitive display 1152 and a keyboard 1154 designed and used inaccordance with an embodiment. The keyboard 1154, when used inaccordance with some embodiments, generates text that is output to thetext display region 1158 at a text insertion location 1160. The term“keyboard” in this application refers to any keyboard that isimplemented on a touch-and tap-sensitive surface, including a keyboardpresented on a touch-sensitive display. The keyboard 1154 shows theletters of the alphabet of the respective language selected by the useron individual keys, arranged in approximately the standard “QWERTY”arrangement found on most keyboards.

In one embodiment, the orientation, location, and size of the keyboard(as well as individual keys) are adaptively changed according to theinput behavior of the user. When the user rests their fingers on thetouch surface 1152 in a certain way, the system moves the keyboard 1154to the location determined by the resting fingers. When the user intendsto actuate a key on the keyboard 1154, they “tap” on the desired key bylifting their finger and striking the surface 1152 with discernibleforce. User taps that occur on areas 1162, 1164 outside of the touchsensor area 1152 are detected by the vibration sensor(s) and may also beassigned to keyboard functions, such as the space bar.

The absence of a touch sensor signal is in effect, a signal with a valueof zero, and when correlated with a tap (or vibration) sensor can beused to uniquely identify a tap location. In one embodiment, thevibration signal for specific regions outside of the touch sensor area1152, such as those indicated at areas 1162, 1164, are unique and storedin a database by the system. When the absence of a touch signal occursin conjunction with a tap event, the system compares the vibrationcharacteristics of the tap with those stored in the database todetermine the location of the external tap. In one embodiment, the lowerouter boundary area 1162 is assigned to a space function, while theright outer boundary area 1164 is assigned to a backspace function.

FIG. 11B is a schematic view representative of an exemplary virtualonscreen keyboard 1170. The keyboard 1170 is divided into two halves: aleft half 1172 and a right half 1174 (as correlates to the left andright hands of the user). The two separate halves 1172, 1174 are notaligned with each other. The eight keys 1178 that are typically restedon by the user are labeled in bold according to which finger istypically used for that key (e.g., L1 represents the index finger of theleft hand, L4 represents the little finger of the left hand, and so on).All other non-home-row keys are indicated by a label showing whichfinger is normally used to type that key using conventional touch-typingtechniques. It should be noted, however, that there are many typingstyles that do not use the finger placements as shown in FIG. 11B, andthose labels are included herein for illustrative purposes only.

The left half of the keyboard 1172 shows all the keys aligned inhorizontal rows, as they would be on a traditional electromechanicalkeyboard. In one embodiment as shown on the right half 1174, thehome-row keys are dispersed along an arc to better fit the normalresting position of the user's four fingers. Non-home-row keys aresimilarly dispersed in accordance with their relative location to thehome-row resting keys. Further, in one embodiment, the size of each keymay also vary in accordance with the statistical likelihood that theuser will select that key (the higher the likelihood, the larger thekey).

FIG. 11C is a schematic view representative of the virtual onscreenkeyboard 1184 that is oriented at an angle in accordance with anembodiment. The user may rest their hands 1190 on the touch-sensitivesurface 1192 of a typical handheld tablet computer 1194 at any locationand orientation that they wish. In this case, the hands are spread apartfurther than normal and oriented at an angle as referenced to thestraight edges of the device 1194. The user initiates an actionindicating a “home-row definition event,” which, may include, but is notlimited to, the following: resting all eight fingers for a short,user-definable period of time; double-tapping all eight fingerssimultaneously on the surface 1192 and then resting them on the surface1192; or pressing down all eight fingers simultaneously as they areresting on the surface 1192. In another embodiment, not all eightfingers are required to initiate a home-row definition event. Forexample, if someone was missing their middle finger, a home-rowdefinition event may be initiated by only three fingers on that hand.Here the user has rested their hands 1190 at an angle on the tabletcomputer 1194, thus causing a processor of the computer 1194 to generateand display the virtual onscreen keyboard 1184 at an angle.

Although some of various drawings illustrate a number of logical stagesin a particular order, stages which are not order dependent may bereordered and other stages may be combined or broken out. Furthermore,in some embodiments, some stages may be performed in parallel and/orsimultaneously with other stages (e.g., operations 315, 320, and 325 inFIG. 3 may all be performed together or substantially in parallel).While some reordering or other groupings are specifically mentioned,others will be apparent to those of ordinary skill in the art, so theordering and groupings presented herein are not an exhaustive list ofalternatives. Moreover, it should be recognized that the stages could beimplemented in hardware, firmware, software, or any combination thereof.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the embodiments to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples and their practical applications, to thereby enable othersskilled in the art to best utilize the embodiments and variousembodiments with various modifications as are suited to the particularuse contemplated.

What is claimed is:
 1. An electronic device, comprising: atouch-sensitive display; one or more processors; memory storing one ormore programs, wherein the one or more programs are configured to beexecuted by the one or more processors, the one or more programsincluding instructions for: displaying a keyboard on the touch-sensitivedisplay; detecting a touch input at the touch-sensitive display; inresponse to detecting the touch input at the touch-sensitive display,determining whether the touch input corresponds to typing or to asliding motion; upon determining that the touch input corresponds totyping: interpreting the touch input as a keyboard event; actuating keyson the keyboard on the touch-sensitive display based on a location ofthe touch input; and upon determining that the touch input correspondsto a sliding motion, interpreting the touch input as a first touchpadevent.
 2. The electronic device of claim 1, further comprising, afterinterpreting the touch input as the first touchpad event, moving acursor on the touch-sensitive display in accordance with the touchinput.
 3. The electronic device of claim 2, further comprising, afterinterpreting the touch input as a first touchpad event, detecting a tapon the touch-sensitive display immediately followed by a dragging motionacross the surface and, in response, interpreting the sliding motion asa second touchpad event, wherein the second touchpad event correspondsto dragging a currently-displayed object across the touch-sensitivedisplay.
 4. The electronic device of claim 1, further comprising, afterinterpreting the sliding motion as a touchpad event, determining whethera path followed by the sliding motion on the touch-sensitive displaytraces out a word and, in accordance with a determination that the pathfollowed by the sliding motion traces out the word, outputting text onthe touch-sensitive display corresponding to the word.
 5. The electronicdevice of claim 1, further comprising: detecting a predefined gesture;and in response to detecting the predefined gesture, ceasing to displaythe keyboard on the touch-sensitive display.
 6. The electronic device ofclaim 5, wherein detecting the predefined gesture includes pressing andholding a predefined number of contacts for a predefined amount of timeon the touch-sensitive display.
 7. The electronic device of claim 5,wherein detecting the predefined gesture includes detecting that a userhas stopped typing.
 8. The electronic device of claim 5, whereindetecting the predefined gesture includes detecting a tap outside of thetouch-sensitive display.
 9. The electronic device of claim 1, whereinactuating keys on the keyboard on the touch-sensitive display based on alocation of the touch input includes determining relative displacementof the touch input relative to a resting position and determiningappropriate keys to actuate based on the relative displacement.
 10. Amethod, comprising: at an electronic device with one or more processors,memory, and a touch-sensitive display: displaying a keyboard on thetouch-sensitive display; detecting a touch input at the touch-sensitivedisplay; in response to detecting the touch input at the touch-sensitivedisplay, determining whether the touch input corresponds to typing or toa sliding motion; upon determining that the touch input corresponds totyping: interpreting the touch input as a keyboard event; actuating keyson the keyboard on the touch-sensitive display based on a location ofthe touch input; and upon determining that the touch input correspondsto a sliding motion, interpreting the touch input as a first touchpadevent.
 11. A method, comprising: at an electronic device with one ormore processors, memory, a touch-sensitive display, and one or moretouch sensors coupled to the touch-sensitive display: displaying aplurality of keys on a keyboard on the touch-sensitive display;detecting, by the one or more touch sensors, a first contact at a firstkey of the plurality of keys on the keyboard; determining a value of asignal corresponding to the first contact; when the value is above afirst non-zero threshold, actuating the first key; and when the value isbetween a second non-zero threshold and the first non-zero threshold,forgoing actuating the first key, wherein the first non-zero thresholdis greater than the second non-zero threshold.
 12. The method of claim11, further comprising: after actuating the first key, determiningwhether the value of the signal corresponding to the first contactremains above the first non-zero threshold for more than a predefinedamount of time; and in accordance with a determination that the value ofthe signal corresponding to the first contact remains above the firstnon-zero threshold for more than a predefined amount of time,continuously actuating the first key.
 13. The method of claim 12,further comprising, while continuously actuating the first key,detecting a decrease in value of the signal and, in accordance with adetermination that the decrease in value exceeds a rate-of-changethreshold, ceasing to actuate the first key.
 14. The method of claim 13,wherein the decrease in value of the signal results in the value of thesignal remaining above the first non-zero threshold.
 15. The method ofclaim 11, wherein the one or more touch sensors include a capacitivetouch sensor, the signal is a touch signal detected by the capacitivetouch sensor, and determining the value of the signal corresponding tothe first contact includes measuring an accumulation of the signalcorresponding to the first contact.
 16. The method of claim 11, furthercomprising, altering the first non-zero and second non-zero thresholdsto accommodate touch signatures associated with different users.
 17. Themethod of claim 11, wherein determining the value of the signalcorresponding to the first contact includes detecting inflection pointsin the signal and measuring the value of the signal at each inflectionpoint.
 18. The method of claim 11, further comprising, when an amplitudeof the signal stabilizes between the first non-zero threshold and thesecond non-zero threshold, forgoing actuating the first key.
 19. Themethod of claim 11, further comprising, when the signal has a risingedge with a rate-of-change that exceeds a first rate-of change thresholdand, within a predefined amount of time thereafter, the signal decreasesin value at a rate-of-change greater than a second rate-of-changethreshold, actuating the first key.
 20. An electronic device,comprising: a touch-sensitive surface; one or more touch sensors coupledto the touch-sensitive display; one or more processors; and memorystoring one or more programs, wherein the one or more programs areconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: displaying a plurality of keys on akeyboard on the touch-sensitive display; detecting, by the one or moretouch sensors, a first contact at a first key of the plurality of keyson the keyboard; determining a value of a signal corresponding to thefirst contact; when the value is above a first non-zero threshold,actuating the first key; and when the value is between a second non-zerothreshold and the first non-zero threshold, forgoing actuating the firstkey, wherein the first non-zero threshold is greater than the secondnon-zero threshold.